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Abstract. In this article we study optimal control problems for sys- 
tems that are affine in one part of the control variable. Finitely many 
equality and inequality constraints on the initial and final state are con- 
sidered. We investigate singular solutions for this class of problems. 
First, we obtain second order necessary and sufficient conditions for weak 
optimality. Afterwards, we propose a shooting algorithm and show that 
the sufficient condition above-mentioned is also sufficient for the local 
quadratic convergence of the algorithm. 

1. Introduction 

The purpose of this paper is to investigate optimal control problems gov- 
erned by ordinary differential equations that are affine in one part of the 
control variable. This class of system includes both the totally affine and 
the nonlinear cases. 

Many models that correspond to this framework can be found in practice 
and, in particular, in the existing literature. Among these we can mention: 
the Goddard's problem [25] in 3 dimensions analyzed in Martinon et al. 

other models concerning the motion of a rocket in Lawden [38J, Bell 
and Jacobson [9], Goh [27J ETJ, Oberle 06], Azimov [8J and Hull [33]; an 
hydrothermal electricity production problem studied in Bortolossi et al. [13] 
and Aronna et al. [6] , the problem of atmospheric flight considered by Oberle 
in [38], and an optimal production process in Cho et al. |17] and Maurer at 
al. HQ. 

The subject of second order optimality conditions for these partially affine 
problems have been studied by Goh in [281 [211 123 ETJ, Dmitruk in [21] . 
Dmitruk and Shishov in [22j, Bernstein and Zeidan [10], and Maurer and 
Osmolovskii |42| . The first works were by Goh, who introduced a change of 
variables in [28] and used it to obtain optimality conditions in [28} 1261 127] , 
always assuming uniqueness of the multiplier. The necessary conditions we 
present imply those by Goh [26] when there is only one multiplier. Recently, 
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Dmitruk and Shishov [22J analysed the quadratic functional associated with 
the second variation of the Lagrangian function and provided a set of nec- 
essary conditions for the nonnegativity of this quadratic functional. Their 
results are consequence of a second order necessary condition we present. In 
[21] Dmitruk proposed, without proof, necessary and sufficient conditions for 
a problem having a particular structure: the affine control variable applies 
to a term depending only on the state variable, i.e. the affine and nonlinear 
controls are 'uncoupled'. This hypothesis is not used in our work. The con- 
ditions established here coincide with those suggested in Dmitruk [21J when 
the latter are applicable. In f 10] , Bernstein and Zeidan derived a Riccati 
equation for the singular linear-quadratic regulator, which is a modification 
of the classical linear-quadratic regulator where only some components of 
the control enter quadratically in the cost function. All of these four articles 
use Goh's Transformation to derive their conditions; we use this transfor- 
mation as well. On the other hand, in [32] Maurer and Osmolovskii gave 
a sufficient condition for a class of problems having one affine control sub- 
ject to bounds and such that it is bang-bang at the optimal solution. This 
structure is not studied here since no control constraints are considered, i.e. 
our optimal control is suppose to be totally singular. 

Regarding second order optimality conditions, we provide a pair of nec- 
essary and sufficient conditions for weak optimality of totally singular solu- 
tions. These conditions are 'no gap' in the sense that the sufficient condition 
is obtained from the necessary one by strengthening an inequality. We do 
not assume uniqueness of multiplier. 

Among the applications of the shooting method to the numerical solution 
of partially affine problems we can mention the articles Oberle [45 } 08] and 
Oberle-Taubert [415]. In these articles the authors use a generalization of 
the algorithm that Maurer [39] suggested for totally affine systems. These 
works present interesting implementations of a shooting-like method to solve 
partially affine control problems having bang-singular or bang-bang solutions 
and, in some cases, running-state constraints are considered. No result on 
convergence is given in these articles. 

In this paper we propose a shooting algorithm which can be also used to 
solve problems with bound on the controls. Our algorithm is an extension 
of the method for totally affine problems in Aronna et al. [7j. We give 
a theoretical support to this method, by showing that the second order 
sufficient condition above-mentioned ensures the local quadratic convergence 
of the algorithm. 

The article is organised as follows. In Section [2] we present the problem, 
the basic definitions and first order conditions. In Section [3] we give the tools 
for second order analysis and establish a second order necessary condition. 
We introduce Goh's Transformation in Section SJ In Section [5] we show 
a new necessary condition, and in Section [6] we give a sufficient one. A 
shooting algorithm is proposed in Section [7J and in Section [8] we prove 
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that the sufficient condition above-mentioned guarantees the local quadratic 
convergence of the algorithm. 

Notations. We denote by ht the value of function h at time t if h is a 
function that depends only on t, and by hi t t the ith component of h evaluated 
at t. Partial derivatives of a function h of (t, x) are referred as Dfh and D x h. 
When dealing with derivatives of higher order we may use the notation of 
type h xx since it is not ambiguous. By R fc we denote the k— dimensional 
real space, i.e. the space of column real vectors of dimension k; and by 
R fe '* its corresponding dual space, which consists of k— dimensional row real 
vectors. By L p (0,T;R fc ) we mean the Lebesgue space with domain equal to 
the interval [0,T] C R and with values in R k . The notation W q ' s (0, T; R k ) 
refers to the Sobolev spaces (see Adams pQ for further details on Sobolev 
spaces). 



2. Statement of the problem and assumptions 

2.1. Statement of the problem. We study the optimal control problem 
(P) given by 



(1) J ■= (po(x ,x T ) -> min, 

m 

(2) x t = ^2vi t tfi(x t ,ut), a.e. on[0,T], 

=o 



(3) r)j(x ,x T ) = 0, for j = 1... ,d v , 

(4) (Pi(xo,x T ) < 0, for i = 1, ... ,d v . 

Here U : R n+ ' R n for i = 0, . . . , m, tpt : R 2n -)■ R for i = 0, . . . , d v , 
rjj : R 2n — > R for j = 1, . . . , cL and we put, in sake of simplicity of notation, 
vq = 1 which is not a variable. The nonlinear control u belongs to U := 
L oo (0,T;R / ), while by V := L^O^R™) we denote the space of affine 
controls v, and X := W 1 ' 00 ^, T; R n ) refers to the state space. When needed, 
we write w = (x, u, v) for a point in W := X x li x V. The hypothesis below 
is considered along all the article. 

Assumption 2.1. All data functions have Lipschitz-continuous second deriva- 
tives. 

A trajectory is an element w E W that satisfies the state equation ([2]). If 
in addition, constraints ([3|) and ([4]) hold, we say that w is a feasible trajectory 
of problem (P). 

Definition 2.2. A feasible trajectory w = (x, u, v) £ W is a weak minimum 
of (P) if there exists e > such that the cost function attains at w its 
minimum in the set of feasible trajectories w = (x, u, v) satisfying 

\\x — x\\ao<£, \\U— uWoo < £, \\v — #||oo < £■ 
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In the sequel, we study a nominal feasible trajectory w = (x, it, v) £ W. 
An element 5w E W is termed feasible variation for w if w + Sw is feasible 
for (P). Take A = (a,/3,p) in R«VH»* x R^>* x W rl ' oo (0, T; R n '*). Define the 
pre-Hamiltonian function 

m 

H[X](x,u,v,t) := p t ^2vifi(x,u), 

i=0 

the terminal Lagrangian function ^ : R 2ra — > R by 

i=o j=l 

and the Lagrangian function 

(5) L[A](w) := ^[A](x ,x T ) + / Pt f v i t fi(x t , Ut) - x t ) di. 

We assume, in sake of simplicity, that whenever some argument of fi, H, £, 
L or their derivatives is omitted, they are evaluated at w. Without loss of 
generality we suppose that 

(6) Pi(xo, xt) = 0, for all % = 1, . . . , d v . 

2.2. Lagrange multipliers. We introduce here the concept of multiplier. 
The second order conditions that we prove in this article are expressed in 
terms of the second variation of the Lagrangian in ([5|) and the set of Lagrange 
multipliers associated with w that we define below. 

Definition 2.3. An element A = (a,(3,p) G R d ^ +1 '* x R d '"* x W 1,oo (0, T; W 1 '*) 
is a Lagrange multiplier associated with w if it satisfies the following condi- 
tions, 

(7) |a| + 101 = 1, 

(8) a = (a ,a>i,...,a dip )>0, 
the function p is solution of the costate equation 

(9) - p t = D x H[X\(xt,u t ,vt,t), 
and it satisfies the transversality conditions 

, w Po = -D X0 £[X}(x ,x T ), 

p T = D XT £[X\(x ,x T ), 
and the stationarity conditions 
m , / D u H[\}(x(t),u(t),v(t),t) = 0, 

( ' I D v H[X}(x(t),u(t),v(t),t) = 0, a,e ' ° n [ ' J ' 

hold true. Denote by A the set of Lagrange multipliers associated with w. 

Recall the following well-known result. 
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Theorem 2.4. If w is a weak minimum, the set A is non empty and com- 
pact. 

Proof. Regarding the existence of a Lagrange multiplier the reader is referred 
to [3 [37], [1U Thm. 2.1]. In order to prove the compactness, observe that p 
may be expressed as a linear continuous mapping of (a, (3). Thus, since the 
normalization (J7J) holds, A is a finite-dimensional compact set. □ 

In view of previous result, note that A can be identified with a compact 
subset of W, where s := + d n + 1. 

Given (xq,u,v) G W 1 x U x V, consider the linearized state equation 

(12) x t = A t x t + E t ut + B t v t , a.e. on [0, T], 

(13) x(0) = xo, 
where 

m m 

(14) A t := ViD x fi{x, u), E t := ^ ViD u fi(x, u), 

1=0 8=0 

and B : [0,T] -»• M nxm {^) such that for every v G M m , 

m 

(15) S t t) := ^2vifi(x t ,u t ). 

i=i 

Here A^ nX m(lR) refers to the space of n x m— real matrices. Hence, the ith. 
column of S is fi(x,u). The solution x of (|12jl -(|13p is called linearized state 
variable. 

2.3. Critical cones. We define now the sets of critical directions associated 
with w, both in the L°°— and the I?— norm. Even if we are working with 
control variables in L°° and hence the control perturbations are naturally 
taken in L°°, the second order analysis involves quadratic mappings and it 
is useful to extend them continuously to I? . 

Set X 2 := W^iO^W 1 ), U 2 := L 2 (0,T;R l ) and V 2 := L 2 (0,T;R m ). Put 
W2 := X 2 x IA2 x V2 for the corresponding product space. Given w G W2 
satisfying (|12p -(|13 p . consider the linearization of the endpoint constraints 
and cost function, 

(16) D7]j(xo,x T )(x ,XT) = 0, for j = 1, . . . ,d v , 

(17) Dipi(x ,XT)(xo,XT) < 0, for i = 0, . . . ,d v . 
Define the critical cones in W and W2 by 

(18) C := {w G W : (T2)-(T3D, dUD-dlZD hold}, 

(19) C 2 := G W 2 : JHJ-jlZJ hold}. 

Lemma 2.5. T/ie critical cone C is a dense subset of C<i- 

In order to prove previous lemma, recall the following technical result (see 
e.g. Dmitruk [20, Lemma 1] for a proof). 
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Lemma 2.6 (on density of cones). Consider a locally convex topological 
space X, a finite-faced cone Z C X, and a linear manifold Y dense in X. 
Then the cone Z PiY is dense in Z. 

Proof, [of Lemma 12. 5j 

Set X :={w £W 2 : ®- $M hold}, Y := {w G W : {12}-® hold}, and 
Z := C2 and apply Lemma 12.61 □ 

3. Second order analysis 

We begin this section by giving an expression of the second derivative 
of the Lagrangian function L, in terms of the derivatives of I and H. We 
denote it by VI. All the second order conditions we present are established 
in terms of either Q, or some transformed form of £1. The main result of the 
current section is the necessary condition in Theorem 13.91 which is applied 
in Section [5] to get Theorem 15.31 

3.1. Second variation. Let us consider the quadratic mapping 

(20) 

fl[X](x,u,v) := ^D 2 £[X\(x ,xt)(x ,xt) 2 
+ [ T [±x T Q[\}x + u T F[X}x + v T C[X]x + ±u T R [X]u + v T K[X]u]dt, 

•J 

where the involved matrices are, omitting arguments, 

(21) Q := H xx , F := H ux , C := H vx , Rq := H uu , K := H vu . 

Recall the following notation: given two functions h : WL N — > K M and k : 
—7- M 1 ', we say the h is a big-0 of k around and denote it by 

h(x) = 0(k(x)), 

if there exists positive constants 5 and M such that |/i(x)| < M|fe(x)| for 
\x\ < 5. It is a small- o if M goes to as \x\ goes to 0. Denote this by 

h(x) = o(k(x)). 

Lemma 3.1 (Lagrangian expansion). Let w = (x,u,v) £ W be a solution 
of and set 5w = (6x, Su, 5v) := w — w. Then for every multiplier A € A, 

(22) L[X](w) = L[X}(w) + n[X}(5x,5u,5v)+T[X](5x,5u,5v)+n{5x,5u,5v), 

where the time variable is omitted in the sake of simplicity, r is a cubic 
mapping given by 

t[X](5x, 8u, Sv) := 
cT 

[H vxx [X](5x,5x,5v) + 2H vux [X](5x,Su, Sv) + H vuu [X](5u, Su, Sv)] dt, 

and 1Z satisfies the estimate 
K(5x,5u,5v) = O(\(Sx ,Sx T )\ 3 ) + (1 + ||v||i)||(fo, ^)||ooC»(||(fe, Su)\$). 
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Proof. Omit the dependence on A for the sake of simplicity. In order to 
achieve the expression ()22f) consider the second order Taylor representations 
below, written in a compact form, 

(23) £(x , x T )=i + D£(5x , 5x T ) + ±D 2 l(5x , 5x T f + O(\(5x ,5x T )\ 3 ), 

(24) fi(x t ,u t ) = hj + DfiiSxuSut) + \D 2 fi{5x t ,5ut) 2 + 0(\\(5x,5u)\\ 3 3 ), 

where, whenever the argument is missing, the corresponding function is 
evaluated on the reference trajectory w. Observe that the transversality 
conditions (fTOj) and the costate equation yield 



(25) D£ (5x , Sxt) = —pa 8x + pt 8xt = / P 

JO 



^2 ViD x fi 5x + 6x 



i=0 



dt. 



Recall the expression of the Lagrangian given in ©. Replacing £(xo, xt) and 
fi(x,u) in © by their Taylor expansions (|23p-(|24p and using the identity 
([231) we get 



L(iu) = L(u))+ / [H u Su + H v Sv]dt + tt(5x, Su, 6v) 
Jo 

+ / [H vxx (5x,5x,5v) +2H vux (5x,5u,5v) + H vuu (5u,5u,5v)] dt 
Jo 

+ O(\(Sx ,6x T )\ 3 ) + ||(<yx,<5u)||oa [ T pjrviO(\\(5x,5u)g)dt. 

J ° i=0 



Finally, to obtain (J22J) use stationarity condition ([lip and the compactness 
of A. □ 

Remark 3.2. The last lemma yields the equality 

(26) n[A](«J) = \D 2 h[\]{w)w 2 . 

3.2. Second order necessary condition. Recall the second order condi- 
tion below. 

Theorem 3.3 (Classical second order necessary condition). If w is a weak 
minimum of problem (P), then 

(27) maxr2[A](a;, u, v) > 0, on C. 

AgA 

A proof of Theorem 13.31 can be found in Osmolovskii [50]. Nevertheless, 
for the sake of completeness, we give a proof here. 

We shall write problem (P) in an abstract form and, therefore, we consider 
the functions 

(28) % : M n x U x V -)■ R, fjj(x , u, v) := rjj(x , xt), 

(29) (pi : R n x U x V — > R, <Pi(xo, u, v) := (fi(xo, xt), 
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where x S W is the solution of ([2]) associated with (xq,u,v). Hence, (P) can 
be written as the following problem in the space M n x U x V, 

min <p (x ,u,v); 

(AP) s.t. fjj(x ,u,v) = 0, for j = 1, . . . ,c^, 

<Pi(x ,u,v) < 0, for 7 = 1,...,^. 

Notice that if w is a weak solution of (P) then (xq,u, v) is a local solution 
of(AP). 

Definition 3.4. We say that the endpoint equality constraints are qualified 
if 

(30) Dfj(x , u, v) is onto from R n xWxVtoR^. 

When (j30|) does not hold, the constraints are noi qualified. 

The proof of Theorem 13.31 is divided in two cases: qualified and not qual- 
ified endpoint equality constraints. In the latter case the condition (p7|) 
follows easily and it is shown in Lemma 13.51 below. The proof for the qual- 
ified case is done by means of an auxiliary problem written in an abstract 
form and its dual. 

Lemma 3.5. // the equality constraints are not qualified then (|27p holds. 

Proof. Observe that since Dfj(xo,u,v) is not onto there exists /3 G M. dr "* 
with |/3| = 1 such that J2j=i PjDfjj(xo,u,v) = and consequently, 

d-n 

^PjDrjj^o^r) = 0. 
i=l 

Set A := (p,a,{3) with p = and a = 0. Then both A and —A are in A. 
Observe that 

d v 

tt[\](x,u,v) = PjD 2 7] j {xo 1 x T ){xQ,x T ) 2 . 
i=i 

Thus, either f2[A](x, u, v) or X](x, u, v) is nonnegative. The desired result 
follows. □ 

Let us now deal with the qualified case. Take a critical direction w = 
(x, u, v) and consider the problem in the variables £ G K and r = (r XQ , r u , r v ) S 
R n x U x V given by 

min £ 

(QP*d) s -t- Dfj(x , u,v)r + D 2 fj(x Q ,u,v)(x ,u,v) 2 = 0, 

D<fi(x ,u,v)r + D 2 ipi(xo,u,v)(x ,u,v) 2 < (, i = 0,...,d v . 

Proposition 3.6. Assume that w is a weak solution of (P) such that the 
endpoint equality constraints are qualified at w. Let w G C be a critical 
direction. Then the problem (QP^) is feasible and has nonnegative value. 
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Proof. Step I. Let us first show feasibility. Since Df](xo,u,v) is onto, there 
exists r S R n x U x V such that the equality constraint in QQPwD is satisfied. 
Set 

C := max {Dipi(x , u, v)r + D 2 i^(x , it, v)(x , u, v) 2 }. 
0<i<d v 

Then ((,r) is feasible for QQPtpD - 

Step II. Let us now prove that QQPtp| ) has nonnegative value. Suppose on 
the contrary that there is (£, r) G E x M. n x U x V feasible for QQPtpP with 
Q < 0. We look now for a family of feasible solutions of (P) that we will 
denote by {r(cj)} (7 . It shall be defined for small positive a and satisfy 

(31) r(cr) -)■ (x ,u,v), ip {r(a)) < ip {x ,u,v). 

The existence of {r(o~)} a will contradict the local optimality of (xq,u,v). 
Consider hence 

r(a) := (xq,u, v) + a(x ,u, v) + \o- 2 r. 
Let < i < d v and observe that 

(Pi{f{a)) = <pi(x ,u,v) + aD(pi(x ,u,v)(xo,u,v) 

(32) + \a 2 [Dtpi(x , u, v)r + D 2 ipi(x , u, v)(x , u, v) 2 ] + o(<r 2 ) 
< <fi(x , U, v) + ifJ 2 C + o(cr 2 ). 

Analogously, 

(33) fl(f(a)) = o(a 2 ). 

Since Dfj(xo,u, v) is onto, there exists r(a) such that ||r(<r) — f((j)|| 00 = o(a 2 ) 
and f){r(a)) = 0. This follows by applying the Implicit Function Theorem 
to the mapping 

(r, a) i-)- fj ((x , u, v) + cr(x , u, v) + \o~ 2 r) = fj{r{o)). 

On the other hand, by taking a sufficiently small in estimate (|32p . we can 
obtain 

(34) <Pi(r(o-)) < ifi(x ,u,v), 

since £ < 0. Hence r(a) is feasible for (AP) and verifies (|3ip . This contradicts 
the optimality of (xq,u, v). We conclude then that all the feasible solutions 
of dQPtpD have ( > 0, and therefore its value is nonnegative. □ 

We shall now go back to the proof of Theorem 13.31 

Proof, [of Theorem 13.3] The not qualified case is covered by Lemma 13.51 
above. Hence, for this proof, assume that ([30]) holds. 

Given w £ C, Proposition 13.61 implies that there cannot exist (£, r) £ 
I x 1" x W x V such that 

!Dfj(x ,u, v)r + D 2 f](xo,u, v)(x ,u, v) 2 = 0, 
D(pi(x , u, v)r + D 2 ipi(x ,u, v)(x ,u, v) 2 < C, % = 0, . . . , dp, 
C<o. 
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Therefore, the Dubovitskii-Milyutin Theorem (see [23]) guarantees the ex- 
istence of (a, 0) G E}+ d v+ d v satisfying, 

dtp dfj 

OiiDtpi(x , u,v) + Y^ PjDfjj(x ,u, v) = 0, 

(35) d9 i= ° j t n 

YaiD 2 (fii(xo,u,v)(xo,u,v) 2 + ^ PjD 2 fjj{x Q , u, v)(x , u, v) 2 > 

i=0 j=l 

Let p be the solution of ([9])- (|10p associated with (a,/3). Let us show that 
A := (a,/3,p) is a Lagrange multiplier for w. In fact, observe that 

dip d'fj 

= ^aiDipi(xo,u,v)(xo,u,v) +^/3jDfjj(x ,u,v)(x ,u,v) 

i=0 j=l 

= D£(xo,x T )(x ,x T ) = -PqXq+ptxt 

= [ {px+px)dt= [ (D u Hu + D v Hv)dt, 
Jo Jo 



where we used (j9|l- (|T0|l and (fT2j) . Hence, necessarily one has 

(36) D u H = 0, D v H = 0, a.e. on [0,T]. 

This implies that A G A. On the other hand, simple computations yield that 
the second line of (f35j) is equivalent to 



(37) n[X](x,u,v) >0, 

and, therefore, the result follows. □ 



Remark 3.7. Observe that condition (|27l) can be extended to the cone C2 by 
the continuity of f2[A] and the compactness of A. 

In the sequel we aim to strengthen previous necessary condition by prov- 
ing that the maximum in (|27|) remains nonnegative when taken in a smaller 
set of multipliers. We shall first give a description of the subset of Lagrange 
multipliers we work with. Set 

(38) H 2 := {(x, u, v) G W 2 : CE2D holds}, 
and consider the subset of A given by 

A # := {A G A : ft [A] is weakly-l.s.c. on H 2 }- 

Lemma 13.81 below provides a characterization of A* and Theorem 13.91 after 
gives a new necessary optimality condition. Recall first the definitions of Rq 
and K given in (|2ip . 

Lemma 3.8. 

(39) A* = {A G A : R [X] h and K[X] = 0}. 
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Theorem 3.9 (Strengthened second order necessary condition). If w is a 

weak minimum of problem (P), then 

(40) max fi[A] (x, u, v) > 0, on Ci. 

xeA* 

In order to prove Lemma 13.81 notice that f2 [A] can be written as the sum 
of two terms: the first one being a weakly-continuous function on the space 
Hi given by 

rT 

(41) (x,u,v) t-> \D 2 i[\]{xo,x T ) 2 + / [\x T Q[X]x + u T F[X]x + v T C[X]x]dt, 

Jo 

and the second one being the quadratic operator 

(42) («,«)•-> f T [hu T Ro[X]u + v T K[X]u]dt. 

Jo 

The weak-continuity of the mapping in (|4ip is a consequence of Hestenes 
[321 Theorem 5.1]. On the other hand, in view of |32^ Theorem 3.2] the 
following characterization holds. 



(43) 



Lemma 3.10. The mapping in (|42p is weakly-lower semicontinuous onlA x 
V if and only if the matrix 

fRo[X] K[X) T " 
\K[X] 

is positive semidefinite a.e. on [0, T]. 

Remark 3.11. The matrix in (I43p is nothing but the second derivative of 
H with respect to the control (u,v). Therefore, the fact that this matrix is 
positive semidefinite is known as the Legendre-Clebsch necessary optimality 
condition for the extremal (w, X) (see e.g. [EJ [2] or Corollary 13.131 below) . 

Notice now that Lemma 13.81 follows from the decomposition given by 
(|4ip - (|42p and previous Lemma 13.101 On the other hand, Theorem 13.91 is a 
consequence of Remark 13 .71 Lemma f3.8l and the following result on quadratic 
forms. 

Lemma 3.12. [191 Theorem 5] Given a Hilbert space H, and ai, 0,%, ■ ■ ■ , a p 
in H, set 

(44) K :={x G H : (oj, x) < 0, for % = 1, . . . ,p}. 

Let M be a convex and compact subset o/R s , and let {Q^ : ifi G M} be a 
family of continuous quadratic forms over H, the mapping ip — > being 
affine. Set := {ifj G M : is weakly-l.s.c. on H} and assume that 

(45) maxQ^x) > 0, for all x G K. 
Then 

(46) max Q*(x) > 0, for all x G K. 
We finish this section with the following Corollary. 
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Corollary 3.13 (Legendre-Clebsch condition). If w is a weak minimum of 
problem (P) with a unique associated multiplier A, then (w,X) satisfies the 
Legendre-Clebsch condition. In order words, the matrix in (I43p is positive 
semidefinite and, consequently, 

(47) Ro[X] h and K[X] = 0. 

Proof. It follows easily from Theorem 13.91 In fact, the inequality in (|40p 
implies that A# ^ 0, and since there is only one multiplier A, it follows that 
A* = {A} and hence (|47p necessarily holds. □ 

4. Goh Transformation 

In this section we introduce a change of variables which consists of a 
linear transformation of (x,u,v). The motivation of this change of variables 
is the following. In previous section we were able to provide a necessary 
condition involving the nonnegativity of the max $7 [A] on 62- The next step 
is to find a sufficient condition and, in order to achieve this, one would 
usually strengthen the inequality (|4"0|) to convert it into a condition of strong 
positivity. But since no quadratic term on v appears in O, the latter cannot 
be strongly positive. The technique we employ to find the desired sufficient 
condition is transforming Q into a new quadratic mapping that may result 
strongly positive on an appropriate transformed critical cone. For historical 
interest, we recall that Goh introduced this change of variables in |28j and 
employed it to derive necessary conditions in [28, 26J. Afterwards, Dmitruk 
in [18] stated a second order sufficient condition for control-affine systems 
(case I = 0) in terms of the uniform positivity of max Q in the corresponding 
transformed space of variables. 

Consider hence the linear system in (|12p and the change of variables 

(48) J y_ tl= J ^ ds ' forte [0,T]. 
I & := x t - B t y t , 

This change of variables can be done in any linear system of differential equa- 
tions, and it is often called Goh's transformation. Observe that £ defined in 
that way satisfies the linear equation 

(49) i = A^+Eu + Bw, e" = S , 
where A and E were given in (|14p . and 

(50) B 1)t := A t B t - -^B t . 
The i—th. column of B\ is given by 

m 
3=0 

where [fi, fj] x := (D x fj)fi - (D x fi)fj. Hence, we make the following hy- 
pothesis of regularity of the controls. 
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Assumption 4.1. The controls u and v are smooth. 

In fact, a procedure of derivation of the controls as a function of the 
state and costate is done in Section [7] afterwards. It is proved that under 
Assumption 17.31 (u,v) can be written as a smooth function of (x, A). 

4.1. Tranformed critical cones. In this paragraph we present the crit- 
ical cones obtained after Goh's transformation. Recall the linearized end- 
point constraints in (fTo]) - (fT7"l) and the critical cones given by (fT8l) - (fT9l) . Let 
(x, u, v) G C be a critical direction. Define (£, y) by transformation (l48j) and 
set h := y T . Note that ([16|1 - ([T7|1 yield 

(51) Dr]j(xo,x T ){Co,CT + B T h) = 0, for j = 1, . . . , d v , 

(52) Z%(x ,^(£o,£t + £t/>) < 0, for i = 0,...,d< p . 

Recall the definition of the linear space W2 in paragraph 12.31 Denote by y 
the space W 1,oo (0,T;R m ), and consider the cones 

(53) V := {(£, u, y, h) G W x M m : y = 0, y T = fc, flU, ffiM 52 ) hold }> 

(54) P 2 := {(£, u, y, /i) G W2 x M m : (09]), (51])-([52]) hold}. 

Remark 4.2. Notice that P consists of the directions obtained by transfer- 
mating the elements of C via transformation (|48j) . 



The next result shows the density of V in Vi . This fact is useful afterwards 
to extend a necessary condition in V to the bigger cone Vi by continuity 
arguments, as it was done for C and Ci in Section [3j 

Lemma 4.3. V is a dense subspace o$Vi W2 x R m — topology. 

Proof. Notice that the inclusion is immediate. In order to prove the density, 
consider the linear spaces 

(55) X := {(£, u, y, h) G W 2 x M m : (05} holds}, 

(56) Y := {(£, u,y,h) G W x R m : y(0) = 0, y(T) = h, and (09]) holds}, 
and the cone 

(57) Z : = {(£, u,y,h)GX : (J5TJ) - ([52]) holds}. 

Notice that Y is a dense linear subspace of X (by Lemma 6 in |22j or Lemma 
8.1 in |5j), and Z is a finite-faced cone of X. The desired density follows by 
Lemma 12.61 □ 

4.2. Transformed second variation. Here we prove that performing the 
Goh's transformation in Q yields the new quadratic operator O-p in variables 
(£, u, y, v, h) defined below and give a new necessary condition in terms of 
O-p. Recall the definitions in (|2ip and set, for A G A*, 

n v [\](t,u,y,v,h):=g[\}(£ ,fr,h)+ [ (^ r Q[MC + u T F[X]C 

(58) Jo 

+ y T M[A]£ + in T i? [A]n + y T J[X]u + iy T i?i[A]y + « T Y[A]y) dt, 
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where 

(59) M := B r Q -C-CA, J := B T F T - CE, 

(60) S := \{CB + (CB) T ), V := \{CB - (CB) T ), 

(61) Ri := B T QB - {CB 1 + (C#i) T ) - S, 

(62) g[X)(Co, Ct, fc) := ^"(Co, Ct + £t/i) 2 + /* T (C t Ct + \S T h). 

Observe that, in view of Assumptions 12.11 and 14.11 an the function defined 
above are continuous in time. Note that easy computations yield 

(63) V ij =p\f i ,f j ]*. 

Theorem 4.4. Let (x,u,v) G H2 (given in (I38p ) and (£,,y) defined by the 
transformation (jUJ). Then 

(64) n[X](x,u,v) = Q v [X](C,u,y,v,y T ). 

Proof. First recall that the term u T i^[A]u in Q[X] vanishes since we are 
taking A E A* and, in view of Lemma 13.81 K[X] = 0. In the remainder of 
the proof we omit the dependence on A for the sake of simplicity. Replacing 
x in the definition of f2 in equation (|20p by its expression in (|48p yields 

n(x,u,v)= \l" (x ,x T )(Zo,fr + B T y T ) 2 + [ + By) 1 Q(| + By) 

Jo 

+ u T F(C + Sy) + « T C(£ + By) + i^i^] dt. 

Integrate by parts the first term containing v in previous equation and use 
(BSD to get 

(66) [ T v T C£dt = [y T C^ - [ T y T {C£ + C(A£ + Eu + B 1 y)}dt. 
Jo Jo 

The decomposition of CB introduced in (|60p followed by an integration by 
parts leads to 

i-T 



(67) 



/ v T CBydt = f v T (S + V)ydt 
Jo Jo 

= ^[y T ^]o + [ T (-±y T Sy + v T Vy)dt. 
Combining ([551) . ([55 11 and ([57) . the identity ([MI) follows. □ 



Finally recall Theorem 13.91 Observe that by performing Goh's transfor- 
mation in ()40p and in view of Remark 14.21 we obtain the following form of 
the second order necessary condition. 

Corollary 4.5. If w is a weak minimum of problem (P), then 

max il-p[A](£, u, y, y, h) > 0, onV. 
aga# 
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5. New second order necessary condition 

We aim to remove the dependence on v in the formulation of the second 
order necessary condition of Corollary 14.51 Note that in (I68j) v appears only 
in the term v T y[A]y. Next we prove that we can restrict the maximum in 
(|68p to the subset of A^ consisting of the multipliers for which V[X) vanishes. 

Denote the convex hull of A* by co A* and let G(co A*) be the subset of 
co A# for which V[X] vanishes, i.e. 

(69) G(coA # ) := {A G coA* : V[X] = 0}. 
The following optimality condition holds. 

Theorem 5.1 (New necessary condition). If w is a weak minimum of prob- 
lem (P), then 

(70) max tt v [A] (f , u, y, y, y T ) > 0, onV. 

XeG(co A#) 

Theorem 15. II is an adaptation of very similar results given in Dmitruk [18J 
and Milyutin [43], that were employed recently in Aronna et al. [5]. The 
proof given in [5] Theorem 4.6] holds for Theorem 15.11 with minor modifica- 
tions and hence we do not include it in the present article. 

Notice that when w has a unique associated multiplier, from Theorem 15. II 
we deduce that G(coA^) is not empty, and since the latter is a singleton, 
we get the corollary below. This corollary is one of the necessary conditions 
stated by Goh in [26] . 

Corollary 5.2. Assume that w is a weak minimum having a unique asso- 
ciated multiplier. Then the following conditions holds. 

(i) V = or, equivalently, the CB is symmetric or, in view of ([63]) . 

PifiJjY = 0, for i,j = 1, . . . ,m, 

where p is the unique associated adjoint state. 

(ii) The matrix 

< 7i > ( j Q 

is positive semidefinite. 

Observe that for A G G(coA#), the quadratic form f2[A] does not de- 
pend on v since its coefficients vanish. We can then consider its continuous 
extension to V2, given by 

n V2 [\]&u,y,h):=g[\}fo,fr,h)+ (" [\V Q[\\l + u 1 E[X}^ 
(72) J 

+ y T M[\]i + in T # [A]n + f ' J[X}u + \f Ri[\)y) dt, 

where the involved matrices were defined in ([14p -(fT5]). (|2ip . and (|59p -([62p. 
From Theorem 15.11 and previous definition, it follows: 
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Theorem 5.3. If w is a weak minimum of problem (P), then 

(73) max Q-p 2 [A](£,u,y, h) > 0, onV 2 - 

AeG(co A#) 

Remark 5.4. The latter optimality condition does not involve v. It is stated 
in the variable (£, u, y, h). 

6. Second order sufficient condition 
for weak minimum 

This section provides a second order sufficient condition for strict weak 
optimality. Its proof is an adaptation of the proof of Theorem 5.5] with 
important simplifications due to the absence of control constraints, but with 
some new difficulties owed to the presence of the nonlinear control variable. 

Define the 7— order by 

(74) j(x ,u,y,h) := \x \ 2 + \h\ 2 + [ (\u t \ 2 + \y t \ 2 )dt, 



for (xo, u, y, h) £ M. n x IA2 x V2 x M m . It can also be considered as a function 
of (xo, u, v) G W 1 x U2 x V2 by setting 

(75) 7(x ,«,v) := i(x ,u,y,y T ), 

with y being the primitive of v defined in (|48p . 
Notation: We write 7 to refer to either 7 or 7. 

Definition 6.1. [7— growth] We say that w satisfies 7— growth condition in 
the weak sense if there exist e, p > such that 

(76) <^( w ) ^ •A' 1 ^) + Pl( x o — xq, u — u,v — v), 
for every feasible trajectory w with \\w — tu||oo < £■ 

Theorem 6.2 (Sufficient condition for weak optimality). (i) Assume that 
there exists p > sitc/i i/iai 

(77) max n V2 [\](£,u,y,h) > pj(^ ,u,y,h), onV 2 . 

XeG(co A#) 

Then w is a weak minimum satisfying 7— growth in the weak sense. 
(ii) Conversely, if w is a weak solution satisfying 7— growth in the weak 
sense and such that a>o > for every X £ G(coA#), then ([77|) 
/or some p > 0. 

Corollary 6.3. if w satisfies (|77p and it /tas a unique associated multiplier, 
then necessarily the matrix in (|78p is uniformly positive definite, i.e. 

(78) ^° Qhpl, «n[0,T], 
where I refers to the identity matrix. 

Remark 6.4. Another consequence of the condition (I77j) is stated in Remark 
18.21 afterwards, where we link it with the strengthened generalized Legendre- 
Clebsch condition. 
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The remainder of this section is devoted to the proof of Theorem l6.21 We 
shall start by establishing some technical results that will be needed for the 
main result. For the lemma below recall the definition of the space %2 in 
®. 

Lemma 6.5. There exists p > such that 

(79) \x \ 2 + ||x||| + \x T \ 2 < pj(xo,u,v), 

for every linearized trajectory (x,u,v) E %2- The constant p depends on 
I \A\ | ooj ||-S||oo; ||ii/||oo and ||-Bi||oo- 

Proof. Throughout this proof, whenever we put pi we refer to a positive 
constant depending on ||^4||oo> ||-B||oo> H-^Hooj and/or ||i?i||oo- Let (x,u,v) £ 
V.2 and (£, y) be defined by Goh's Transformation (pl8j) . Thus (£,u,y) is 
solution of (|49p . Gronwall's Lemma and Cauchy- Schwartz inequality yield 



(80) Heiloo < Pl {\t \ 2 + + \\y\\l) 1/2 < Pi-f(xo,u,y,yT) 1/2 , 

with pi = /9i(||j4||i,||£'|| 00 ,||Si|| 00 ). This last inequality together with the 
relation between £ and x provided by (|48p imply 

(81) \\x\\ 2 < Uh + ||-B||oo||y||2 < P27{x ,u,y,y T ) 1/2 , 

for p2 = P2{pi, 1 1 -B | |oo)- On the other hand, (HHj) and estimate ([801) lead to 
\xt\ < |£r| + ll-BHoolyrl < pi~f(xQ,u,y,y T ) l/2 + ||-B||oo|yr|- 



Then, in view of Young's inequality '2o6 < a? + b 21 for real numbers o, b, 
one gets 

(82) |x T | 2 < P3l(x , u, y, y T ), 

for some p3 = p3(pi, ||-B||oo). The desired estimate follows from (f8lT) and 
(H2|). □ 

Notice that Lemma 16.51 above gives an estimate of the linearized state in 
the order 7. The following result shows that the analogous property holds 
for the variation of the state variable as well. 

Lemma 6.6. Given C > 0, there exists p > such that 

(83) \o~xq\ 2 + ||<5x||2 + |<5xt| 2 < pj(Sxq,5u,Sv), 

for every (x,u,v) solution of the state equation ([2]) having \\v\\2 < C, and 
where 5w := w — w. The constant p depends on C, \\B\loo, H-BHoo and the 
Lipschitz constants of f{. 

Proof. In order to simplify the notation we omit the dependence on t. Con- 
sider (x,u, v) solution of ([2]) with ||u||2 < C. Let 5w := w — w, 5y := / 5v, 
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and £ := Sx — B5y, with B given in (fT5|) and y t := L v s ds. Note that 

m m 

(84) i :° 

i=0 

where Vq = 1. In view of the Lipschitz-continuity of /j, 

(85) «)-/»(*,«)! <L(|fe| + |H) <^(lel + || J B|U<fy| + IH), 
for some L > 0. Thus, from (|84p it follows 



lei < + Piioci^i + ih)(i + M) + u^un 

= L{\£\(1 + |«|) + ||B||oo|^| + H + ||B||oo|<5l/||«| + |HI«I> + ll^ll< 

Applying Gronwall's Lemma and Cauchy-Schwartz inequality to previous 
estimate yields 

(86) lieiU < pxdCol + ll%lli + IIHIi + INI2IMI2 + IIHI2IMI2), 

for pi = pi(L,C, ||-B||oo, Halloo)- Hence, since ||<5x|| 2 < ||£||2 + ||-B||oo||<fy||2, 
by previous estimate and Cauchy-Schwartz inequality, the desired result 
follows. □ 

Finally, the following lemma gives an estimate for the difference between 
the variation of the state variable and the linearized state. 

Lemma 6.7. Consider C > and w = (x,u,v) £ W a feasible trajectory 
with \\w — w\\oo < C. Set (Sx, u,v) := w — w and let x be the linearization of 
x associated with (Sxq,u,v). Define 

(87) := Sx - x. 
Then, •& is solution of the differential equation 

m m 

■& = ViD x fi(x, ujti + X ViDfi(x, u)(Sx, u) + C, 
i=Q i=l 
^o = 0, 

where the remainder C, satisfies the estimates 

(89) HClloo < Pi, ||C|| 2 <P2 7, 

where pi,P2 depend on C, H-D 2 /^ and the Lipschitz constant of D 2 f. If in 
addition, \\u\\2 + ||v||2 — > 0, the following estimates for § hold 

(90) Moo < o(Vt), ||tf|| 2 < o(Vt). 

Proof. Let us begin by observing that the variation of the state variable 
satisfies the differential equation 

m m 

(91) Sx = ^2vi[fi(x,u) - fi{x,u)] +^2vifi(x,u). 

i=0 i=l 
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Consider the following Taylor expansions for fa, 
(92) 

fi(x,u) = fi(x,u) + Dfi(x,u)(8x,u) + \D 2 fi(x,u)(5x,u) 2 + p \(Sx,u)\ 3 , 

where po is a function of the Lipschitz constant of D 2 fi. Combining (|9ip 
and fl§2D yields 



(93) 5x = ^2viDfi(x,u)(5x,u) + ^Vifi(x,u) + (, 

i=Q i=l 

with the remainder being given by 

m 

(94) C== ^«<[D 2 /<(^«)(fo,«) 2 + Po|(5a!,«)| 3 . 

i=Q 

The linearized equation (|12[) together with (|93p lead to (|88p . and, in view 
of (|94p . it can be seen that the estimates in (|89j) hold. Applying Gronwall's 
Lemma in (|88p . and using Cauchy-Schwartz inequality afterwards lead to 



< P3 



^2 ViDfi(x, u)(Sx, u) + C 



<P4[||v|| 2 (||fe|| 2 + ||u|| 2 ) + ||C||2], 



for some positive ps,p4 depending on C and HD/Hoq. Finally, using the 
estimate in Lemma [6.6l and (|89p just obtained, the inequalities in (|90p follow. 

□ 

In view of Lemmas 13.11 16.51 16.61 and 16.71 we can justify the following tech- 
nical result that is an essential point in the proof of the sufficient condition 
of Theorem \~ 



Lemma 6.8. Let w G W be a feasible solution. Set (6x,u,v) := w — w, and 
x its corresponding linearized state, i.e. the solution of (|12p -(|13 p associated 
with (5xo,u,v). Assume that \\w — u>||oo — > 0. Then 

(95) £[A]H = L[X](w) + Q[\]{x,u,v) +0(7). 

Proof. Omit the dependence on A for the sake of simplicity. Recall the 
expansion of the Lagrangian function given in Lemma 13.11 Notice that by 
Lemma \QM, L(io) = L(w) + U(Sx, u, v) + 0(7). Hence, 

(96) L(w) = L(w) + Q(x,u,v) + AQ + o(~f), 

with AJ7 := U(Sx, u, v) — Q(x, u, v). The next step is using Lemmas 16.51 16.61 
and 16.71 to prove that 

(97) AO = 0(7). 

Note that Q(a, a) — Q(b, b) = Q[a + b, a — b), for any bilinear mapping Q, 
and any pair a, b of elements in its domain. Set 1? := Sx — x as it is done in 
Lemma 16.71 Hence, 

rT 

AO = U"((5x +x , 5x t +xt), (0, tf T ))+ / [USx+x) 1 Qti+u 1 EdW C0\dt 

Jo 
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The estimates in Lemmas 16.51 and 16.61 yield Att = J f T C'ddt + 0(7). Inte- 
grating by parts in the latter expression and using (|90j) lead to 



v T CMt = [y T C$\l - [ T f(C$ + C&)dt = 0(7), 
Jo 

and hence the desired result follows. □ 

Proof, [of Theorem 16.2] We shall prove that if (|77p holds for some p > 0, 
then w satisfies 7— growth in the weak sense. By the contrary assume that 
the 7— growth condition (fT6|) is not satisfied. Consequently, there exists a 
sequence of feasible trajectories {w k } converging to w in the weak sense, 
such that 

(98) J(w k ) < J(w) + o{ lk ), 

with 5w k :=w k -w and j k := r y(5x kt o,u k ,v k ). Let (£ k ,u k ,y k ) be the trans- 
formed directions defined by (08]). We divide the remainder of the proof in 
two steps. 

(I) First we prove that the sequence given by 

(99) {£,k,u k ,y k ,h k ) := u k , y k , hk)/y/7k 

contains a subsequence converging to an element (£,u,y,h) of V2 m 
the weak topology, i.e. (u k ,y k ) — 1 (u, y) in the weak topology of 
U2 x V2 and h k ) — > (£, h) in the strong sense of X 2 x W a . 
(II) Afterwards, employing the latter sequence and its weak limit, we 
show that (|77)1 together with ([98]) lead to a contradiction. 

We shall begin by Part (I). For this we take an arbitrary Lagrange mul- 
tiplier A in A*. By multiplying the inequality ([98]) by cxq, and adding the 
nonpositive term 

dm dfj 

(100) ^2aiipi(x kfi ,x k)T ) + ^2f3jr]j(x kfi ,x k ,T), 
i=o 3=1 

to its left-hand side, the inequality follows 

(101) L[X](w k )<L[X}(w) + o( 7k ). 

Let us now recall the expansion (|95p given in Lemma 16.81 Note that the 
elements of the sequence (£fc,o, u k ,y k , h k ) have unit W 1 XU2 x V2 x R m — norm. 
The Banach-Alaoglu Theorem (see e.g. |14l Theorem III. 15]) implies that, 
extracting if necessary a subsequence, there exists (t;o,u,y,h) £ R™ x W 2 x 
V 2 x R m such that 

(102) | fcj0 -)• £0, «fc w, yfc y, fi* -> h, 

where the two limits indicated with — are taken in the weak topology of IA2 
and V2, respectively. The solution of equation (|49]) associated with (Co,u,y) 
is denoted by £, which is the limit of £ k in ^2- F° r the aim of proving 
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that (£, u, v, h) belongs to V2, we shall check that the initial-final conditions 
(|5ip - (l52l ) are verified. For each index < i < dm, one has 

(103) D(pi(x ,XT)(^o,^T + B T h) = lim D(pi(x ,x T ) 

k — ^00 

In order to prove that the right hand-side of (|103j) is nonpositive, we consider 
the following first order Taylor expansion of function ipi around (xq,xt) '■ 

<Pi(xk,o,x kjT ) = ipi{x ,x T ) + Dipi(x ,x T )(5x kfi ,5x ktT ) +o(\(Sx kj0 ,Sx k:T )\). 

Previous equation and Lemmas 16.51 and 16,71 imply 

Vi{Xk,Q,Xk,T) = tpi{x ,x T ) + Dipi(xo,x T ){x kfl ,x^ T ) +o{y/jk~). 

Thus, the following approximation for the right hand-side in (|103|) holds, 

(104) D(pi(xa,x T ) ' = ! '—= yo(l). 

V \fik ) y/lk 

Since w k is a feasible trajectory, it satisfies (jl|) and, therefore, equations 
(fT03D and (fTDll) yield, for 1 < i < d v , D(pi(x , x T )(|o, £r + B^fi) < 0. For 
j = use inequality to get the corresponding inequality. Analogously, 

(105) DrijixoMdo, £t + B T h) = 0, for j = 1, . . . , <V 

Thus (£,u,y,h) satisfies (j5Tj) - (l52j) . and hence it belongs to T^- 

Let us deal with Part (II). Notice that from (|95|) and (I101D we get 

(106) ^p 2 W(6fe>«fc>yjfe>^fc) < cC 1 )^ 
and thus 

(107) liminf f^ 2 [A](4, %, 2/fc, h k ) < 0. 
Consider the subset of G(co A*) defined by 

(108) Af' p := {A G G(coAf ) : ftp 2 [A] - p 7 is weakly l.s.c. on ^ 2 x M m }. 
By applying Lemma 13.121 to the inequality (f77|) one has 

(109) max n P2 [X](^,u,y,h) > frf(( ,u,y,h), on TV 

AGAf 

We shall take A S A*' p that attains the maximum in (|109p for the direction 
(£, u, y, /i). Hence we get 

(110) ° ~ n V2 [\](i,u,y,~h) - frf(io,u,y,h) 

< \\m.m{ k ^^Q. V2 [\](£ k ,u k ,y k ,h k ) - py(£k,o,u k ,y k ,h k ) < -p, 

since 0-p 2 [A] — pj is weakly-l.s.c, "y(Ck,o,u k ,y k ,h k ) = 1 for every k and in- 
equality (|107p holds. This leads us to a contradiction since p > 0. Therefore, 
the desired result follows. 

(ii) Let us now prove the second statement. Assume that w is a weak 
solution satisfying 7— growth in the weak sense for some constant p' > 0, 



Xk,0, X k ,T 
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and such that ao > for every multiplier A G G(coA^). We consider the 
modified problem 

J(w) — p >n ((w — w) — > min, 

(P) s.t. ©-gD 

and rewrite it in the Mayer form 

J(w) - p'(\x - x \ 2 + \yr - Vt\ 2 + ^i,t + 7t 2 ,t 



mm, 



s.t. © - ®, 

y = v, 

TTl = (U - U) 2 , 

7T2 = (y- y) 2 , 

y Q = 0, 7Ti j0 = 0, 7r 2;0 = 0. 

We aim to apply the second order necessary condition of Theorem 15.31 to 

at the point (w = w, y = y, it\ = 0,7r2 = 0). Simple computations 
show that at this solution each critical cone of ()53[) is the projection of 

the corresponding critical cone of and that the same holds for the set 



of multipliers. Furthermore, the second variation of (|P[) evaluated at a 
multiplier A G G(coA#) is given by 

(HI) n-p 2 [X](i,u,y,y T ) - a p'j{x ,u,y,y T ), 

where A G G(co A^) is the corresponding multiplier for problem ()53|) . Hence, 
the necessary condition in Theorem 15 . 31 implies that for every (£, u, v, h) G V2 
there exists A G G(coA#) such that 

n-p 2 [\](t=,u,y,y T ) ~ a p'"/(x ,u,y,y T ) > 0. 

Setting p := min G ( coA #) aop' > yields the desired result. This completes 
the proof of the theorem. □ 



7. Shooting algorithm 

The purpose of this section is to present an appropriate numerical scheme 
to solve the problem given by equations CE])-©, which we denote (SP). No- 
tice that no inequality endpoint constraints are considered. More precisely, 
we investigate the formulation and the convergence of an algorithm that 
approximates an optimal solution provided an initial estimate. 

We shall consider an hypothesis concerning the endpoint conditions. With 
this end recall Definition 13.41 The following holds throughout the rest of the 
article. 

Assumption 7.1. The endpoint equality constraints are qualified or, equiva- 
lently, the derivative of 77 at (xq,u,v) is onto. 
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It is a well-known result that in this case w is normal and has a unique 
associated multiplier (see e.g. Pontryagin et al. |52j). Therefore, without 
loss of generality, we can consider olq = 1. The unique multiplier associated 
with w is denoted by A = ($,p). 

7.1. Optimality system. In what follows we use the first order optimality 
conditions (|lip to provide a set of equations from which we can determine 
w. We obtain an optimality system in the form of a two-point boundary value 
problem (TPBVP). 

We shall recall that for the case where all the control variables appear 
nonlinearly (m = 0), the classical technique is using the stationarity equa- 
tion 

(112) H u [X](w) = 0, 

to write u as a function of (x, A) (this is done in e.g. [16], [40l [12], [54] ) . One is 
able to do this by assuming, for instance, the strengthened Legendre-Clebsch 
condition 

(113) H uu [X](w) y 0. 

The latter condition comes from strengthening the inequality in the neces- 
sary optimality condition mentioned in Remark 13.111 which is verified by w 
in view of Corollary 13.131 In this case, due to the Implicit Function The- 
orem, we can write u = U[X](x) with U being a smooth function. Hence, 
replacing the occurrences of u by U[X](x) in the state and costate equations 
yields a two-point boundary value problem. 

On the other hand, when the system is affine in all the control variables 
(I = 0), we cannot eliminate the control from the equation H v = and, 
therefore, a different technique is employed (see e.g. [39 ] 147 ] IBT j \12 \ 17] [54]) . 
The idea is to consider an index 1 < i < m, and to take d Mi H v /dt Mi to 
be the lowest order derivative of H v in which Vi appears with a coefficient 
that is not identically zero. Kelley [34] : Goh [28] [27], Kelley et al. [35] 
and Robbins [53J proved that Mj is even when the investigated extremal is 
normal. This implies that H v depends only on x and A and, consequently, 
it is differentiable in time. Thus the expression 

(114) H v [X](w) = 

is well-defined. The control v can be retrieved from (|114p provided that, for 
instance, the strengthened generalized Legendre-Clebsch condition 

r) FT 

(115) _^[ A ](^0 

holds (see Goh (27] [30] El]). In this case, we can write v = V[X](x) with 
V being differentiable. By replacing v by V[A](x) in the state-costate equa- 
tions, we get an optimality system in the form of a boundary value problem. 

In the problem studied here, where / > and m > 0, we aim to use both 
equations (|112j) and ()114f) to retrieve the control (u, v) as a function of the 
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state x and the multiplier A. We next describe a procedure to achieve this 
elimination that was proposed in Goh [30\ 13 lj . Let us show that H v can be 
differentiated two times in the time variable, as it was done in the totally 
affine case. Observe that (|112p may be used to write i as a function of 
(A, w). In fact, in view of Corollary 13. 131 

(116) H uv = 0, 

and hence the coefficient of v in H u is zero. Consequently, 

(117) H u = H u [X](x,u,v,ii) =0 

and, if the strengthened Legendre-Clebsch condition (|113p holds, u can be 
eliminated from (|117p yielding 

(118) ii = T[X](x,u,v). 
Take now an index i = 1, . . . , m and observe that 

d m 

(119) = H V . = —pfi = p^v j [f j J i } x (x,u) + H ViU ti = p[fo,f i } x (x,u), 

3=0 

where Corollary 15.21 and (|116p are used in the last equality. Therefore, 
H v = H v [X](x, u). We can then differentiate one more time H v , replace the 
occurrence of u by T in (|118p and obtain (|114p as it was desired. See that 
(|114p together with the boundary conditions 

(120) H v [\](w T ) = 0, 

(121) H v [\](w o ) = 0, 

guarantee the second identity in the stationarity condition (|lip . 
Notation: Denote by (OS) the set of equations consisting of ([2])-([3]), (|7|), 
©-(HUD, (HI2]), (fTTil) and the boundary conditions f[T20l)- (fT2T|) . 

Remark 7.2. Instead of (|120p - (|12ip . we could choose another pair of end- 
point conditions among the four possible ones: H v> q = 0, H v> t = 0, H V) q = 
and H Vj t = 0, always including at least one of order zero. The choice we 
made will simplify the presentation of the result afterwards. 

Observe now that the derivative of the mapping (u, v) i->- ( gj. ) is given 



by 

(122) J := I dH v dH v 



du dv 

On the other hand, if (|113p and (|115p are verified, J is definite positive 
and consequently, nonsingular. In this case we may write u = U[\\(x) and 
v = V[\](x) from (fTT2l) and (jlTij) . Thus (OS) can be regarded as a TPBVP 
whenever the following hypothesis is verified. 
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Assumption 7.3. The conditions (|113p and (|1 15[) hold along w. 

Summing up we get the following result. 

Proposition 7.4 (Elimination of the control). If Assumption \7.,°J\ holds, 
then one has 

u = U[X](x), v = V[X](x), 
for smooth functions U and V. 

Remark 7.5. When the linear and nonlinear controls are uncoupled, this 
elimination of the controls is much simpler. An example is shown in Oberle 
[48j where a nonlinear control variable can be eliminated by the stationar- 
ity of the pre-Hamiltonian, and the remaining problem has two uncoupled 
controls, one linear and one nonlinear. 

The rest of this article is very close to what was done in Aronna et al. 
[7J. The main difference between the totally affine case and the mixed case 
treated here lies on the derivation of the system (OS). The proof of the 
convergence in Section [8] is an extension of the proof of Theorem 5 in [7] . 
The presentation here is then more concise, and the reader is referred to the 
mentioned article for further details. 

7.2. The algorithm. The aim of this section is to present a numerical 
scheme to solve system (OS). In view of Proposition 17.41 we can define the 
following mapping. 

Definition 7.6. Let S : W 1 x R*+*»>* = : D(<S) R d ^ x R^+2m,* be the 
shooting function given by 

T](x ,X T ) 

Po + D xo £[X](xo,x T ) 

(123) (x ,p ,P) =: v ^S{y):=\ pr - D XT £[X](x ,x T ) 

H v [\](w T ) 

H v (w ) 

where (x,p) is a solution of ([2]).(}9]). (jll2p . (|114p with initial conditions xq and 
Po, and A := (p,(3), and where the occurrences of u and v were replaced by 
u = U[X](x) and v = V[X](x). 

Note that solving (OS) consists of finding z> G D(5) such that 

(124) S(P) = 0. 

Since the number of equations in (|124p is greater than the number of un- 
knowns, the Gauss-Newton method is a suitable approach to solve it. The 
shooting algorithm we propose here consists of solving the equation (|124p by 
the Gauss-Newton method. A more extensive description of this algorithm 
is presented in [7j. There it is observed that the method is applicable pro- 
vided that 5'(i>) is one-to-one, with v := {xq,pq, (3). Furthermore, since the 
right hand-side of system (I124p is zero, it converges locally quadratically if 
the function S has Lipschitz continuous derivative. The latter holds true 
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here given the regularity hypotheses on the data functions (in Assumption 
12. ip . This convergence result is stated in the proposition below. See e.g. 
Fletcher [24j for a proof. 

Proposition 7.7. If S'{p) is one-to-one then the shooting algorithm is lo- 
cally quadratically convergent. 

8. Convergence of the shooting algorithm: 
application of the second order sufficient condition 

The main result of this last part of the article is the theorem below that 
gives a condition guaranteeing the quadratic convergence of the shooting 
method near an optimal local solution. 

Theorem 8.1. Suppose that w is such that (I77p holds. Then the shooting 
algorithm is locally quadratically convergent. 

The idea is linking the sufficient condition (|77p to the derivative S'{v). 
Notice that (|77p is expressed in the variables after Goh's Transformation, 
while S is in the original variables. The procedure to achieve Theorem 18.11 
has three stages that are described in the paragraphs 18. 1\ [8.21 and \8. 31 below. 
The proof of[8J]is at the end oflO 

Remark 8.2. In view of a result in |27^ Section 4.8] the positive definite- 
ness in (|78p implies both (|113p and (|115p . Therefore, in Theorem 18 .1\ the 
Assumption 17.31 is guaranteed by the condition (|77p . 

8.1. Linearization of (OS). We write the linearized system associated 
with (OS), which gives the derivative of S. A definition of linearized dif- 
ferential algebraic system can be found in e.g. Kunkel-Mehrmann [36] or 
Aronna et al. [7J. We denote by Lin J 7 the linearization of function J 7 , i.e. 

(125) Lin^ \ (M) (Ct,a t ) := a° t )(( t , a t ), 

The technical result below will simplify the computation afterwards. Its 
proof is immediate (or see |36j). 

Lemma 8.3 (Commutation of linearization and differentiation). Given Q 
and T as in the previous definition, it holds: 

(126) — Lin^ = Lin ^-Q, Lin J 7 = Lin ^-F. 

y ' dt dt dt dt 

Recall the definitions in (I14p . (I15p and (12ip . Notice that, since H v = pB, 

(127) Lin-f^ = pB + x T C T . 

Here whenever the argument of a function is missing, assume that it is 
evaluated on (w, X). The linearization of system (OS) at point (x,u, v, X) 
consists of the linearized state equation (|12p with endpoint condition (|16p . 
the linearized costate equation 

(128) -Pt = p t A t + xjQt + ujE t + vJC u a.e. on [0, T], 



SINGULAR SOLUTIONS IN OPTIMAL CONTROL 



27 



with boundary conditions 
(129) 



(130) 



Po 



PT 



3=1 



(&0,x T ) 



T 



Dy+^D 2 xoXT e+^2^D XTVj 



3=1 



(x Q ,X T ) 



and the algebraic equations 



(131) 
(132) 
(133) 
(134) 










Lin H u = pD + x 1 E 1 + u 1 R , 
d 2 



Lin H v 



dt 2 



(pB + x^C 1 ), a.e. on [0,T], 



(Lin H V ) T = ptBt + x^C? , 
(Lin H v ) = 



dt 



t=0 



{pB + x'C 



where we used equation (|127l) and the commutation property of Lemma [8 .31 
Observe that (fLl2l) - (fl34"|) and Lemma El yield 

(135) = Lin H v =pB + x T C T , a.e. on [0,T]. 

Notation: denote by (LS) the set of equations consisting of (fl~2l) . (fT6|) . 
(fl^ - (fl34jl . 

Proposition 8.4. The differential 5'(P) is one-to-one if the only solution 
°f (I12p . (]128p . (|131j) . (I132p wii/i i/ie initial conditions (xoiPo) = and u>ii/i 
/3 = is (x,u,v,p) = 0. 

8.2. Auxiliary linear-quadratic problem. Now we introduce the follow- 
ing linear-quadratic control problem in the variables (£,,u,y,h). Denote by 
(LQ) the problem given by 

(136) n V2 (£, u, y, h) min, 

(137) <m,m, 

(138) h = 0. 

Here u and y are the control variables, £ and /i are the state variables, and 
U-p 2 is the quadratic mapping defined in (|72|) associated with A. 

Let x an d X/i be the costate variables corresponding to £ and h, respec- 
tively. Note that the qualification hypothesis in Assumption 17.11 implies 
that {Dr]j(xo, xt)}^=i are linearly independent. Hence any weak solution 
(£, u,y,/i) of (LQ) has a unique associated multiplier \ L ® := (x,Xh,P ) 
solution of the system that we describe next. The pre-Hamiltonian for (LQ) 
is 



(139) 



Hh^QZ + u T Ei + y T Mi + \u T R Q u + f Ju + \f Riy), 
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and the endpoint Lagrangian is given by 



(140) e LQ [\ LQ ](Zo, £t, hr) ■= s(fo, fr, kr) + E /^ 9 ^(£o,£t + B T h T ). 

3=1 

The costate equation for x is 

(141) - i t = D^n[X LQ ] =xA + fQ + u T E + y T M, 
with the boundary conditions 



(142) 



Xo 



-o LQ 
Po 



UD\ 2 l + (£ T + B T h) T Dl QXT i - /3f Q D xoVj , 



(143) 

X T = D iT l L Q[\ LC i] 



P L o Q 



&D*£+& + BTh) T I>* 



+ h T C T + j:f =1 l3f Q D XT r )j . 



For the costate variable Xh we get the equation and endpoint conditions 

(144) i h = 0, 

(145) Xhfl = 0, 

(146) Xh,T = D- h £ L ^[X L % 



Hence, %h = and thus (|146[) yields 
(147) 

dr, 

= ^ Q f^"D^B T + (f T + B T h) T {D 2 x2 £Br + )1 pf^D XTVj B T . 



i=i 



The stationarity with respect to the control (it, y) implies 

(148) 
(149) 



= Hu = xD + t T E T + u T R + y T J, 
= Uy = xBi+ fM T + u T J T + y T i?L 



Notation: Denote by (LQS) the set of equations consisting of ()137|) - (jl38|) . 



Note that if the uniform positivity (|77j) holds, then (LQ) has a unique op- 
timal solution (£, u, y, /i) = 0. Besides, in view of Corollarv l6.3t the strength- 
ened Legendre-Clebsch condition holds for (LQ) at (£, u, y, h) = 0. Hence, 
the unique local optimal solution of (LQ) is characterized by its first order 
optimality system (LQS). This leads to the following result. 

Proposition 8.5. If the uniform positivity in (|77p holds, the system (LQS) 
has a unique solution (£, u, y, h) = 0. 
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8.3. The transformation. Given (x, u, v,p, 0) G W x W 1,co x R d ^'*, define 
(150) 

Vt ■= J o v s ds, f := x- By, x -=P + y T C, Xh '■= 0, h := y T , Pf Q ■= 

Lemma 8.6. The one-to-one linear mapping 

(151) {x,u,v,pj) ^ (£,u,y,h,x,Xh,P LQ ) 

defined by f| 1 50 [) converts each solution of (LS) into a solution of (LQS). 

Proof. It is an easy extension of Lemma 7.1 in [TJ. □ 

We shall now go back to the convergence Theorem 18.11 

Proof, [of Theorem 18. 1] Let (x,u,v,p,j3) be a solution of (LS), and let 
(£, u, y, h, x, Xh, P L ®) be defined by the transformation in (|150p . Hence we 
know by Lemma IHU1 that (£, u, y, h, x, Xh, fi L ®) is solution of (LQS). As it 
has been already shown in Proposition 18.51 condition (J77J) implies that the 
unique solution of (LQS) is 0. Hence (^, u, y, h, x, Xh, ft L ®) = and thus 
(x,u,v,p, f3) = 0. Conclude that the unique solution of (LS) is 0. This 
yields the injectivity of S' at z>, and hence the result follows. 

□ 

Remark 8.7 (The shooting algorithm for the control constrained case). We 
claim that the formulation of the shooting algorithm above and the proof of 
its local convergence can be done also for problems where the controls are 
subject to bounds of the type 

(152) < ut < 1, < v t < 1, a.e. on [0, 1]. 
This extension should follow the procedure in Section 8 of [7J. 

9. Conclusion 

We studied optimal control problems in the Mayer form with systems that 
are affine in some components of the control variable. A set of 'no gap' nec- 
essary and sufficient second order optimality conditions is provided. These 
conditions apply to a weak minimum and do not assume the uniqueness of 
multipliers. For qualified solutions, we proposed a shooting algorithm and 
proved that its local convergence is guaranteed by the sufficient condition 
ab ove- ment ioned . 

There are several issues in this direction of investigation that remain open. 
For instance, one can think of the study of other type of minimum, like 
Pontryagin or strong. Other possible task is the optimality of bang-singular 
solutions, that had not yet been deeply looked into but show to be use- 
ful in practice. Therefore, the results presented can be pursued by many 
interesting extensions. 
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